最近，大多数手写的数学表达识别（HMER）方法采用编码器 - 编码器网络，该网络直接从具有注意机制的公式图像中直接预测标记序列。但是，此类方法可能无法准确读取具有复杂结构的公式或生成长的标记序列，因为由于写作样式或空间布局的差异很大，注意结果通常是不准确的。为了减轻此问题，我们为HMER提出了一个名为Counting-Aware-Aware网络（CAN）的非常规网络，该网络共同优化了两个任务：HMER和符号计数。具体而言，我们设计了一个弱监督的计数模块，该模块可以预测每个符号类的数量，而无需符号级别的位置注释，然后将其插入HMER的典型基于注意力的编码器模型。在基准数据集上进行的实验验证了关节优化和计数结果既有益于纠正编码器模型的预测误差，又可以始终如一地胜过最先进的方法。特别是，与HMER的编码器模型相比，提议的计数模块引起的额外时间成本是边缘的。源代码可从https://github.com/lbh1024/can获得。

本文介绍了一种新的普通话 - 英语代码转换语音识别的语料库 - 塔尔奇语料库，适用于培训和评估代码转换语音识别系统。滑石乐谱来自TAL教育小组中真正的在线在线一对一的英语教学场景，其中包含大约587个小时的语音采样16 kHz。据我们所知，滑石科目是世界上标签最大的普通话 - 英语密码开关开源自动语音识别（ASR）数据集。在本文中，我们将详细介绍录制过程，包括捕获设备和语料库环境的音频。并且滑石场可以根据允许许可证免费下载。我们使用滑石乐谱，在两个流行的语音识别工具包中进行ASR实验，以制造包括ESPNET和WENET在内的基线系统。在滑石粉中比较了两个语音识别工具包中的混合错误率（MER）性能。实验结果表明，音频记录和转录的质量是有希望的，基线系统是可行的。

对象异常的检测对于工业过程至关重要，但是由于难以获得大量有缺陷的样本以及现实生活中无法预测的异常类型，因此无监督的异常检测和定位尤为重要。在现有的无监督异常检测和定位方法中，基于NF的方案取得了更好的结果。但是，两个子网（复杂函数）$ s_ {i}（u_ {i}）$和$ t_ {i}（u_ {i}）在nf中通常是多层的perceptrons，需要从2D扁平至1D，破坏了特征图中的空间位置关系并丢失空间结构信息。为了保留并有效提取空间结构信息，我们在这项研究中设计了一个复杂的函数模型，该模型具有交替的CBAM嵌入在堆叠的$ 3 \ times3 $全卷积中，该卷积能够保留并有效地在标准化流程模型中提取空间结构信息。 MVTEC AD数据集的广泛实验结果表明，Cainnflow基于CNN和Transformer Backbone网络作为特征提取器达到高级准确性和推理效率，并且Cainnflow可在MVTEC广告中获得$ 98.64 \％的像素级AUC $ 98.64 \％\％。

由于缺乏注释的病理图像，转移学习是数字病理领域的主要方法。基于Imagenet数据库的Pre培训的神经网络通常用于提取“从架子”特征中，以预测组织类型实现巨大成功，分子特征和临床结果等。我们假设使用组织病理学图像进行微调的模型可以进一步改善特征提取，下游预测性能。我们使用了100,000个注释的他的结肠直肠癌（CRC）的图像斑块到FINetune通过TwoStep方法预先训练的Xcepion模型。通过：（1）来自CRC的图像的图像分类，从CRC的图像进行了比较了从FineTuned Xception（FTX2048）模型和图像预测（IMGNET2048）模型的特征; （2）预测免疫基因表达和（3）肺腺癌（Luad）基因突变.FiveFold交叉验证用于模型性能评估。来自FFTuned FTX2048的提取特征在于与基于Imagenet数据库的Xcepion直接从架子特征预测CRC的螺栓类型的螺栓类型的精度显着更高。特别是，FTX2048显着提高了87％至94％的基质的精度。类似地，来自FTX2048的特征促进了免疫烯丙基蛋白拉德转录组表达的预测。对于具有与图像诱导的脑状有关系的基因，特征FGROM FERUNED模型的预测是对大多数基因的预测。从FTX2048中携带灌注，改善了拉德中9个最常见的突变基因中的5个突变的预测。

联合学习是一个分布式机器学习机制，本地设备在中央服务器的编排中协作培训共享全局模型，同时保留所有私有数据分散。在系统中，传输模型参数及其更新而不是原始数据，因此通信瓶颈已成为一个关键挑战。此外，近期的较大和更深层次的机器学习模型也在将它们部署到联邦环境中的困难造成更多困难。在本文中，我们设计了一个联合的两阶段学习框架，即在设备上使用切割层增强了原型联合学习，并使用基于符号的随机梯度下降与大多数投票方法进行模型更新。剪切图层在设备上学习本地原始数据的信息和低维表示，有助于减少全局模型参数并防止数据泄漏。基于符号的SGD与大多数投票方式进行模型更新，也有助于缓解通信限制。凭经验，我们表明我们的系统是一种有效和隐私，保留联合学习计划和适用于一般应用方案的诉讼。

The task of referring video object segmentation aims to segment the object in the frames of a given video to which the referring expressions refer. Previous methods adopt multi-stage approach and design complex pipelines to obtain promising results. Recently, the end-to-end method based on Transformer has proved its superiority. In this work, we draw on the advantages of the above methods to provide a simple and effective pipeline for RVOS. Firstly, We improve the state-of-the-art one-stage method ReferFormer to obtain mask sequences that are strongly correlated with language descriptions. Secondly, based on a reliable and high-quality keyframe, we leverage the superior performance of video object segmentation model to further enhance the quality and temporal consistency of the mask results. Our single model reaches 70.3 J &F on the Referring Youtube-VOS validation set and 63.0 on the test set. After ensemble, we achieve 64.1 on the final leaderboard, ranking 1st place on CVPR2022 Referring Youtube-VOS challenge. Code will be available at https://github.com/Zhiweihhh/cvpr2022-rvos-challenge.git.

Referring image segmentation aims to segment the target object described by a given natural language expression. Typically, referring expressions contain complex relationships between the target and its surrounding objects. The main challenge of this task is to understand the visual and linguistic content simultaneously and to find the referred object accurately among all instances in the image. Currently, the most effective way to solve the above problem is to obtain aligned multi-modal features by computing the correlation between visual and linguistic feature modalities under the supervision of the ground-truth mask. However, existing paradigms have difficulty in thoroughly understanding visual and linguistic content due to the inability to perceive information directly about surrounding objects that refer to the target. This prevents them from learning aligned multi-modal features, which leads to inaccurate segmentation. To address this issue, we present a position-aware contrastive alignment network (PCAN) to enhance the alignment of multi-modal features by guiding the interaction between vision and language through prior position information. Our PCAN consists of two modules: 1) Position Aware Module (PAM), which provides position information of all objects related to natural language descriptions, and 2) Contrastive Language Understanding Module (CLUM), which enhances multi-modal alignment by comparing the features of the referred object with those of related objects. Extensive experiments on three benchmarks demonstrate our PCAN performs favorably against the state-of-the-art methods. Our code will be made publicly available.

Transformer-based models have been widely demonstrated to be successful in computer vision tasks by modelling long-range dependencies and capturing global representations. However, they are often dominated by features of large patterns leading to the loss of local details (e.g., boundaries and small objects), which are critical in medical image segmentation. To alleviate this problem, we propose a Dual-Aggregation Transformer Network called DuAT, which is characterized by two innovative designs, namely, the Global-to-Local Spatial Aggregation (GLSA) and Selective Boundary Aggregation (SBA) modules. The GLSA has the ability to aggregate and represent both global and local spatial features, which are beneficial for locating large and small objects, respectively. The SBA module is used to aggregate the boundary characteristic from low-level features and semantic information from high-level features for better preserving boundary details and locating the re-calibration objects. Extensive experiments in six benchmark datasets demonstrate that our proposed model outperforms state-of-the-art methods in the segmentation of skin lesion images, and polyps in colonoscopy images. In addition, our approach is more robust than existing methods in various challenging situations such as small object segmentation and ambiguous object boundaries.

Differentiable Search Indices (DSIs) encode a corpus of documents in the parameters of a model and use the same model to map queries directly to relevant document identifiers. Despite the strong performance of DSI models, deploying them in situations where the corpus changes over time is computationally expensive because reindexing the corpus requires re-training the model. In this work, we introduce DSI++, a continual learning challenge for DSI to incrementally index new documents while being able to answer queries related to both previously and newly indexed documents. Across different model scales and document identifier representations, we show that continual indexing of new documents leads to considerable forgetting of previously indexed documents. We also hypothesize and verify that the model experiences forgetting events during training, leading to unstable learning. To mitigate these issues, we investigate two approaches. The first focuses on modifying the training dynamics. Flatter minima implicitly alleviate forgetting, so we optimize for flatter loss basins and show that the model stably memorizes more documents (+12\%). Next, we introduce a generative memory to sample pseudo-queries for documents and supplement them during continual indexing to prevent forgetting for the retrieval task. Extensive experiments on novel continual indexing benchmarks based on Natural Questions (NQ) and MS MARCO demonstrate that our proposed solution mitigates forgetting by a significant margin. Concretely, it improves the average Hits@10 by $+21.1\%$ over competitive baselines for NQ and requires $6$ times fewer model updates compared to re-training the DSI model for incrementally indexing five corpora in a sequence.

Prompt tuning has been employed as an efficient way to adapt large vision-language pre-trained models (e.g. CLIP) to various downstream tasks in data-limited or label-limited settings. Nonetheless, visual data (e.g., images) is by default prerequisite for learning prompts in existing methods. In this work, we advocate that the effectiveness of image-text contrastive learning in aligning the two modalities (for training CLIP) further makes it feasible to treat texts as images for prompt tuning and introduce TaI prompting. In contrast to the visual data, text descriptions are easy to collect, and their class labels can be directly derived. Particularly, we apply TaI prompting to multi-label image recognition, where sentences in the wild serve as alternatives to images for prompt tuning. Moreover, with TaI, double-grained prompt tuning (TaI-DPT) is further presented to extract both coarse-grained and fine-grained embeddings for enhancing the multi-label recognition performance. Experimental results show that our proposed TaI-DPT outperforms zero-shot CLIP by a large margin on multiple benchmarks, e.g., MS-COCO, VOC2007, and NUS-WIDE, while it can be combined with existing methods of prompting from images to improve recognition performance further. Code is released at https://github.com/guozix/TaI-DPT.